This paper explores multi-task learning (MTL) for face recognition. We answerthe questions of how and why MTL can improve the face recognition performance.First, we propose a multi-task Convolutional Neural Network (CNN) for facerecognition where identity classification is the main task and pose,illumination, and expression estimations are the side tasks. Second, we developa dynamic-weighting scheme to automatically assign the loss weight to each sidetask, which is a crucial problem in MTL. Third, we propose a pose-directedmulti-task CNN by grouping different poses to learn pose-specific identityfeatures, simultaneously across all poses. Last but not least, we propose anenergy-based weight analysis method to explore how CNN-based MTL works. Weobserve that the side tasks serve as regularizations to disentangle thevariations from the learnt identity features. Extensive experiments on theentire Multi-PIE dataset demonstrate the effectiveness of the proposedapproach. To the best of our knowledge, this is the first work using all datain Multi-PIE for face recognition. Our approach is also applicable toin-the-wild datasets for pose-invariant face recognition and achievescomparable or better performance than state of the art on LFW, CFP, and IJB-Adatasets.
展开▼
